Skip to main content
Cornell University

In just 5 minutes help us improve arXiv:

Annual Global Survey
We gratefully acknowledge support from the Simons Foundation, member institutions, and all contributors. Donate
arxiv logo > cs > arXiv:2305.14902

Help | Advanced Search

Computer Science > Computation and Language

(cs)
[Submitted on 24 May 2023 (v1), last revised 10 Mar 2024 (this version, v2)]

Title:M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection

Authors:Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
View a PDF of the paper titled M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection, by Yuxia Wang and 14 other authors
View PDF HTML (experimental)
Abstract:Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. However, this has also raised concerns about the potential misuse of such texts in journalism, education, and academia. In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse. We first introduce a large-scale benchmark \textbf{M4}, which is a multi-generator, multi-domain, and multi-lingual corpus for machine-generated text detection. Through an extensive empirical study of this dataset, we show that it is challenging for detectors to generalize well on instances from unseen domains or LLMs. In such cases, detectors tend to misclassify machine-generated text as human-written. These results show that the problem is far from solved and that there is a lot of room for improvement. We believe that our dataset will enable future research towards more robust approaches to this pressing societal problem. The dataset is available at this https URL.
Comments: 41 pages
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2305.14902 [cs.CL]
  (or arXiv:2305.14902v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2305.14902
arXiv-issued DOI via DataCite

Submission history

From: Yuxia Wang [view email]
[v1] Wed, 24 May 2023 08:55:11 UTC (51 KB)
[v2] Sun, 10 Mar 2024 01:04:48 UTC (4,382 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection, by Yuxia Wang and 14 other authors
  • View PDF
  • HTML (experimental)
  • TeX Source
license icon view license
Current browse context:
cs.CL
< prev   |   next >
new | recent | 2023-05
Change to browse by:
cs

References & Citations

  • NASA ADS
  • Google Scholar
  • Semantic Scholar
export BibTeX citation Loading...

Bookmark

BibSonomy logo Reddit logo

Bibliographic and Citation Tools

Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)
  • About
  • Help
  • contact arXivClick here to contact arXiv Contact
  • subscribe to arXiv mailingsClick here to subscribe Subscribe
  • Copyright
  • Privacy Policy
  • Web Accessibility Assistance
  • arXiv Operational Status